Stat260/CS294: Randomized Algorithms for Matrices and Data

Here, we will consider one approach for extending the ideas underlying the least-squares algorithm we discussed in class to non-tall matrices. Let A ∈ Rn×d matrix, where both n and d are large, and where rank(A) = k exactly, and let B ∈ Rn×t. Consider the problem minX∈Rn×t‖AX −B‖, where ‖ · ‖ is a unitarily-invariant matrix norm. The solution to this problem is Xopt = AB, and here we consider approximating Xopt with the solution of a sketched problem, where the sketching matrix Z ∈ Rn×r. The sketching matrix Z could be a random sampling or random projection matrix, but for now assume only that rank(ZU) = k, i.e., the r× k matrix ZU had full rank, where the n× k matrix U consists of the top k left singular vectors of A, i.e., all the singular vectors associated with nonzero singular values. Recall that the min-length solution to the sketched problem can be expressed as